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1  Summary 


During  the  performance  period  of  the  grant  we  developed  the  proposed  goal-oriented  probability  den¬ 
sity  function  method  based  on  the  Mori-Zwanzig  formulation  and  applied  it  to  stochastic  PDEs  such  as 
advection-reaction  and  Burgers  equations  (see  section  1.2.1).  We  also  developed  new  algorithms  to  solve 
high-dimensional  probability  density  function  (PDF)  equations  (see  section  1.2.2)  and  stochastic  domain 
decomposition  techniques  to  propagate  uncertainty  across  heterogeneous  domains.  These  research  efforts 
resulted  in  two  papers  published  in  the  Proceedings  of  the  Royal  Society  of  London  A,  two  papers  under  re¬ 
view  during  performance  period  (one  published  now  and  one  in  press)  and  one  book  chapter  (now  in  press). 
The  main  findings  can  be  summarized  as  follows:  The  Mori-Zwanzig  approach  has  the  potential  to  over¬ 
come  well-known  limitations  encountered  in  stochastic  simulations  of  high-dimensional  random  systems, 
in  particular  the  curse  of  dimensionality  and  the  lack  of  regularity  of  the  solution.  This  comes  at  the  price 
of  solving  complex  integro-differential  PDEs  -  the  Mori-Zwanzig  equations  -  whose  computability  relies  on 
either  analytical  approximations  or  data-driven  approaches.  We  investigated  the  accuracy  of  analytical  tech¬ 
niques  based  Kubo-Van  Kampen  operator  cumulant  expansions  for  Langevin  equations  driven  by  fractional 
Brownian  motion  and  other  noises,  stochastic  advection-reaction  equations  and  stochastic  Burgers  equation. 
In  the  context  of  the  Burgers  equation,  we  studied  numerically  one-  and  two-point  MZ-PDF  equations  and 
computed  the  statistical  properties  of  random  shock  waves  generated  by  high  dimensional  random  initial 
conditions  and  random  additive  forcing  terms.  We  also  addressed  the  problem  of  computing  the  numerical 
solution  to  high-dimensional  PDF  equations.  To  this  end,  we  developed  three  different  classes  of  new  al¬ 
gorithms:  the  first  one  is  based  on  separated  series  expansions,  resulting  in  a  sequence  of  low-dimensional 
problems  that  can  be  solved  recursively  and  in  parallel  by  using  alternating  direction  (ADI)  methods.  The 
second  class  of  algorithms  relies  on  a  truncation  the  Bogoliubov-Born-Green-Kirkwood-Yvon  (BBGKY) 
hierarchy  of  coupled  probability  density  function  equations.  The  third  class  of  algorithms  is  based  on  high¬ 
dimensional  model  representations,  e.g.,  ANOVA  and  probabilistic  collocation  methods.  A  common  feature 
of  all  these  approaches  is  that  allow  us  to  compute  the  solution  of  high-dimensional  PDF  equations  via  a 
sequence  of  low-dimensional  problems.  This  is  a  very  important  question  which  will  find  its  way  in  many 
related  works.  For  instance,  we  are  currently  using  the  alternating  direction  Galerkin  method  we  developed 
for  high-dimensional  PDF  equations  in  the  context  of  stochastic  optimal  control,  i.e.,  optimal  control  under 
uncertainty  of  nonlinear  dynamical  systems,  and  plasma  dynamics  (solution  to  the  full  Boltzmann  equation). 

1.1  Published  Papers  and  Book  Chapters 

1.  D.  Venturi,  H.  Cho  and  G.  E.  Karniadakis,  ‘“Mori-Zwanzig  approach  to  uncertainty  quantification”, 
Springer  Handbook  on  Uncertainty  Quantification,  2016  In  Press 

2.  H.  Cho,  X.  Yang,  D.  Venturi  and  G.  E.  Karniadakis,  “Numerical  methods  for  propagating  uncertainty 
across  heterogeneous  domains”,  SIAM  J.  Sci.  Comput.,  2015,  In  Press. 

3.  H.  Cho,  D.  Venturi  and  G.  E.  Karniadakis,  “Numerical  methods  for  high-dimensional  probability 
density  function  equations”,  J.  Comput.  Phys.,  2016,  315,  pp.  817-837. 

4.  H.  Cho,  D.  Venturi  and  G.  E.  Karniadakis,  “Statistical  analysis  and  simulation  of  random  shocks  in 
stochastic  Burgers  equation”,  Proc.  R.  Soc.  A,  2014,  470  (2171),  pp.  1-21. 

5.  D.  Venturi  and  G.  E.  Karniadakis,  “Convolutionless  Nakajima-Zwanzig  equations  for  stochastic  anal¬ 
ysis  in  nonlinear  dynamical  systems”,  Proc.  R.  Soc.  A,  2014,  470  (2166),  pp.  1-20. 

6.  D.  Venturi  and  G.  E.  Karniadakis,  “Convolutionless  Nakajima-Zwanzig  equations  for  stochastic  anal¬ 
ysis  in  nonlinear  dynamical  systems”,  Proc.  R.  Soc.  A,  2014, ,  470  (2166)  Supplement,  pp.  1-13. 
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t  =  1  t  =  2 


Figure  1:  Stochastic  Burgers  equation.  One  realization  of  the  velocity  field  computed  by  using  adaptive 
discontinuous  Galerkin  methods  (left).  Time  snapshots  of  the  one-point  PDF  obtained  by  solving  the  MZ- 
PDF  equation  (1). 


1.2  Research  Accomplishments 

In  this  section  we  would  like  to  summarize  some  of  the  research  accomplishment  we  have  published  in  the 
papers  listed  above,  in  particular  in  Ref.  [3]  and  Ref.  [4] 


1.2.1  Mori-Zwanzig  Equations  for  Burgers  Turbulence 

The  MZ-PDF  approach  allows  us  to  systematically  eliminate  degrees  of  freedom  from  nonlinear  random 
system,  to  obtain  formally  exact  equations  for  quantities  of  interest.  In  the  context  of  randomly  forced 
Burgers  turbulence,  this  allows  us  to  derive  exact  equations  for  the  one-point  PDF  of  the  velocity  field  by 
formally  integrating  out  random  noise  of  small  amplitude  or  small  correlation  length.  The  result  of  such 
integration  can  be  represented  as  a  perturbation  expansion  in  powers  of  a  suitable  coupling  constant,  e.g., 
noise  amplitude.  The  result  is 


dpu(t) 

dt 


=L0pu(t)  +  a  (f(x,  t))  dp^  + 


da 


a 


Pu(t)  + 


(1) 


where  pu(t)  =  p(x,  t,  a)  is  the  one-point  one-time  PDF  of  the  velocity  field  (a  is  the  phase-space  coordinate 
associated  with  u),  and  Lq  is  the  linear  operator 


Lt)  —  — 


fa  ,  d  d 
/  da— - a- 

'  —  OO 


dx  dx 


(2) 


In  Figure  2  we  compare  the  PDF  dynamics  obtained  by  solving  Eq.  (1)  with  Monte  Carlo  simulation.  It 
is  seen  that,  as  we  increase  the  amplitude  a  of  the  forcing  term,  the  first-  and  second-order  truncations  of 
equation  (1)  lose  accuracy.  However,  if  we  include  more  terms  in  the  perturbation  series  we  can  get  as  close 
as  we  like  to  the  MC  results  -  provided  we  re-normalize  the  perturbation  series. 

In  summary,  the  MZ-PDF  framework  allows  us  to  study  the  statistical  properties  of  the  solution  to  the 
stochastic  Burgers  equation  subject  to  random  space-time  perturbations  and  random  initial  conditions.  This 
includes  random  shock  waves  in  both  physical  and  probability  spaces  (see  the  results  in  Ref.  [4]).  The  math¬ 
ematical  methods  and  algorithms  we  developed  for  the  Burgers  equation  can  be  applied  to  other  randomly 
forced  conservation  laws,  potentially  leading  to  new  insights  in  high-dimensional  stochastic  dynamics  and 
more  efficient  computational  techniques. 
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Figure  2:  Stochastic  Burgers  equation.  One-point  PDF  of  the  velocity  field  at  x  =  ir  for  exponentially 
correlated,  homogeneous  (in  space)  random  forcing  processes  with  correlation  time  0.01  and  amplitude 
<7  =  0.01  (left  plot)  and  a  =  0.1  (right  plot).  Shown  are  results  obtained  from  MC,  and  from  two  different 
truncations  of  the  MZ-PDF  equation  (1). 


1.2.2  Numerical  Methods  for  High-Dimensional  PDF  equations 


In  this  section  I  would  like  to  briefly  discuss  the  research  accomplishments  in  the  context  of  numerical 
methods  for  high-dimensional  probability  density  function  equations.  In  particular,  I  would  like  to  highlight 
a  new  class  of  adaptive  algorithms  based  on  separated  series  expansions  (SSE)  which  yield  a  sequence 
of  low-dimensional  problems  that  can  be  solved  recursively  and  in  parallel  by  using  alternating  direction 
methods.  The  mathematical  details  of  the  new  algorithms  are  described  in  Ref.  [3]  above.  Given  a  high¬ 
dimensional  PDF  evolution  equation 


(3) 


where  L  is  a  linear  differential  operator,  we  look  for  a  representation  of  the  solution  p  =  p(x\ . ....  xn.  t)  as 
a  summation  of  separable  functions,  i.e., 


R  n 

p{x±,...,xn,t)  =  J2ptr)^YlPir)^xi) 
r= 1  j= 1 


(4) 


(r)  (r) 

In  this  way,  we  can  determine  the  unknown  functions  p)  and j))  ’  (1, ...,  n)  by  using  an  alternating-direction 
weighted  residual  formulation,  -  in  particular  an  alternating  direction  Galerkin  method.  In  this  way,  the 
computational  cost  of  solving  high-dimensional  PDF  equations  scales  linearly  with  the  dimension  of  the 
PDF  and  the  algorithm  can  be  parallelized  with  asynchronous  updates  -  we  are  currently  working  on  this. 
We  applied  the  alternating-direction  Galerkin  method  to  a  variety  of  PDF  equations,  including  prototype 
equations  in  high-dimensional  parameter  spaces,  e.g. 

m  =  -  (sin(t)  fj  5(Fn) sin({k + 1)-T),’t)  I  ■  <5) 

This  equation  involves  two  phase  variables  (x  and  a),  and  m  parameters  (/>/,.).  Equation  (5)  represents  the 
one-point  PDF  formulation  of  a  randomly  forced  advection  problem  in  one  spatial  dimension.  To  compute 
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Figure  3:  PDF  equation  (5):  series  expansion  modes  p(r\x,  t,  a)  at  t  =  2. 


the  solution  to  such  equation  we  look  for  a  series  expansion  in  terms  of  separated  functions 

R  m 

p(x,t,a,bi,...,bm)  =  y^p(r)(x,f,a)  ]~[  h^jbj),  (6) 

r=  1  j=  1 

and  determine  the  unknowns  jR  (x,  t )  and  /tjj  ^  by  solving  at  most  two-dimensional  linear  problems  at  each 
time  step.  In  Figure  3  we  show  the  modes  j/r>  (x,  f)  ( r  =  1, 10)  we  obtain  at  t  =  2  The  L2  error  of  the 
separated  solution  (6)  relative  to  an  analytical  solution  is  shown  in  Figure  4  versus  the  number  of  parameters 
and  for  different  separation  ranks  R.  The  main  feature  of  the  numerical  methods  we  developed  for  high¬ 
dimensional  PDF  equations  is  that  they  allow  us  to  compute  their  solution  by  solving  a  sequence  of  low¬ 
dimensional  problems.  This  will  find  its  way  in  many  different  applications,  ranging  from  the  Boltzmann 
equation  to  the  Fokker-Plank  and  Liouville  equations.  In  particular,  we  are  currently  using  the  separated 
series  expansion  method  for  optimal  control  of  the  Liouville  equation,  i.e.,  optimal  control  under  uncertainty 
of  nonlinear  dynamical  systems. 

1.3  International  Collaborations 

During  the  reporting  period  of  grant  I  invited  Lucia  Parussini,  a  young  assistant  professor  from  the  depart¬ 
ment  of  Mechanical  Engineering  at  University  of  Trieste,  to  join  the  Division  of  Applied  Mathematics  at 
Brown  University  from  September  8  to  October  4,  2014  and  from  May  13  to  June  14,  2015.  Lucia’s  visit  was 
funded  by  Italy/US  science  and  technology  exchange  program  sponsored  by  AFOSR.  For  technical  reason 
those  fundings  were  added  to  the  AFOSR  grant  I  am  reporting  on.  Lucia  collaborated  with  me,  Prof.  Karni- 
adakis  and  other  members  of  the  research  group  at  Brown  in  developing  new  methods  addressing  the  curse 
of  dimensionality  forward  uncertainty  quantification  (UQ)  problems.  Specifically,  she  studied  a  multivariate 
Gaussian  process  regression  approach  that  allows  us  to  infer  quantities  of  interest,  including  random  func¬ 
tions  and  fields,  in  physical  models  governed  by  stochastic  partial  differential  equations  (SPDEs).  We  have 
results  on  stochastic  Burgers,  and  Navier-Stokes  equations  indicating  that  the  new  method  is  accurate  and 
can  achieve  speed-up  factors  of  up  to  several  orders  of  magnitude  relative  to  classical  UQ  approaches.  This 
was  made  possible  by  combining  the  output  of  variable-fidelity  stochastic  simulations  (variable-fidelity  in 
both  models  and  probability  space)  in  a  Bayesian  inference  process  that  relies  on  recursive  co-kriging.  Lucia 
brought  to  the  Division  of  Applied  Math  at  Brown  the  expertise  and  the  mathematical  tools  developed  by 
the  research  group  on  optimization  of  Prof.  Carlo  Poloni  at  University  of  Trieste.  Carlo  is  the  owner  of  the 
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Figure  4:  PDF  equation  (5):  Relative  L2  errors  of  the  separated  series  expansion  of  the  solution  relative  to  a 
benchmark  solution.  Shown  are  results  for  different  number  of  parameters  m  and  different  separation  ranks 
R  in  (6).  It  is  seen  that  the  accuracy  of  the  separated  expansion  method  mainly  depends  on  the  separation 
rank  rather  than  on  the  number  of  random  variables. 

ESTECO  company  that  developed  the  multi-objective  and  multi-disciplinary  optimization  software  package 
modeFrontier.  The  collaboration  with  Lucia  triggered  AFOSR  in  the  context  of  the  Italy/US  international 
initiative  is  still  active  and  will  continue  in  the  years  to  come. 

1.4  Invited  Lectures,  Conferences  and  Workshops 

Hereafter,  I  provide  a  list  of  all  invited  lectures  and  talks  I  gave  from  the  tentative  start  date  of  the  project 
(Nov  1,  2013)  to  the  final  date  (July  31,  2015).  Note:  not  all  lectures  and  conferences  listed  here  were 
financially  supported  by  AFOSR  (see  the  cost  breakdown  section  1.5),  but  all  of  them  involve  the  research 
work  I  performed  in  the  context  of  the  AFOSR  project.  I  acknowledged  support  from  AFOSR  in  all  of  them. 

Invited  Lectures  in  Universities 

1.  D.  Venturi,  “Information  fusion  in  stochastic  models  of  physical  systems”,  Stanford  University ,  Palo 
Alto  (CA),June  8,  2015. 

2.  D.  Venturi,  “Multi-fidelity  stochastic  modeling,  simulation  and  information  fusion  in  physical  sys¬ 
tems”,  Arizona  State  University,  Tempe  (AZ),  Mar.  25-27 ,  2015. 

3.  D.  Venturi,  “Coarse-graining  high-dimensional  random  systems:  the  probability  density  function  ap¬ 
proach”,  University  of  Massachusetts  Darthmouth,  Dartmouth  (MA),  Mar.  24,  2015. 

4.  D.  Venturi,  “A  new  paradigm  for  variable-fidelity  stochastic  modeling  and  information  fusion  in  phys¬ 
ical  systems”,  University  of  California  Santa  Cruz,  Santa  Cruz  (CA),  Feb.  2nd,  2015. 

5.  D.  Venturi,  “Spectral  properties  of  random  correlation  operators”,  University  of  Massachusetts  Amherst, 
Amherst  (MA),  Dec.  10th,  2014. 

6.  D.  Venturi,  “Computing  with  uncertainty:  coarse-graining  stochastic  dynamical  models  and  PDF 
equations”,  University  of  California  Santa  Cruz,  Santa  Cruz  (CA),  Nov.  24th,  2014. 
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7.  D.  Venturi,  “Reduced-order  methods  for  probability  density  function  equations”,  University  of  Mas¬ 
sachusetts  Amherst,  Amherst  (MA),  Nov.  4th,  2014. 

8.  D.  Venturi,  “Recent  developments  on  reduced-order  methods  for  high-dimensional  kinetic  equations”, 
North  Carolina  State  University ,  Raleigh  (NC),  Sep.  16th,  2014. 

9.  D.  Venturi,  “State  variable  versus  probability  density  function  methods  in  the  stochastic  modeling  of 
nonlinear  dynamical  systems”,  University  of  Delaware,  Delaware,  Apr.  17th,  2015 

10.  D.  Venturi,  “Statistical  analysis  and  simulation  of  random  shock  waves  in  Burgers  turbulence”,  George 
Washington  University,  Washington  DC,  Apr.  7th,  2014. 

11.  D.  Venturi,  “A  new  framework  for  stochastic  analysis  in  large  scale  simulations  based  on  convolution¬ 
less  Nakajima-Zwanzig  PDF  equations”,  EPFL,  Lausanne  (Switzerland),  Dec.  19th,  2013. 

12.  D.  Venturi,  “Convolutionless  Nakajima-Zwanzig  PDF  equations  for  stochastic  analysis  in  large  scale 
simulations”,  Massachusetts  Institute  of  Technology,  Boston  (MA),  Dec.  3rd,  2013. 

Invited  Talks  in  Conferences  and  Workshops 

1.  D.  Venturi,  Dimension  reduction  in  kinetic  equations:  Mori-Zwanzig  formulation,  BBGKY  hierar¬ 
chies  and  effective  propagators,  Houghton  Conference  on  Non-Equilibrium  Statistical  Mechanics, 
ICERM,  Providence  (RI),  May  4-5th,  2015. 

2.  D.  Venturi,  Dimension  reduction  through  the  Mori-Zwanzig  formulation,  SIAM  Workshop  on  Dimen¬ 
sion  Reduction ,  Penn  State  University,  State  College  (PA),  Mar.  21-24,  2015. 

3.  D.  Venturi,  A  multi-fidelity  data-driven  paradigm  for  information  fusion  in  physical  and  biological 
systems,  Pacific  Northwest  National  Laboratory,  Richland  (WA),  March  13,  2015. 

4.  D.  Venturi,  Variable-fidelity  stochastic  modeling  and  calibration  of  multi-physics  systems,  Schlumberger- 
Doll  research  center,  Boston  (MA),  Dec.  1, 2014. 

5.  D.  Venturi,  Statistical  analysis  and  simulation  of  random  shock  waves  in  scalar  conservation  laws, 
APS-DFD  2014,  San  Francisco  (CA),  Nov.  23-25,  2014. 

6.  D.  Venturi,  MZ-PDF  methods  for  UQ,  AFOSR  Program  Review  in  Computational  Math.,  Arlington 
(VA),  July  29-31,2014. 

7.  D.  Venturi,  A  new  framework  for  stochastic  analysis  in  large  scale  simulations  based  on  goal-oriented 
probability  density  function  methods,  (Keynote  Lecture)  WCCM/ECCOMAS,  Barcelona,  July  20-25, 
2014. 

8.  D.  Venturi,  MZ-PDF  methods  for  stochastic  analysis  in  nonlinear  dynamical  systems,  SIAM  AN  14, 
Chicago,  July  7-11,  2014 

9.  D.  Venturi,  Goal-oriented  probability  density  function  methods  for  uncertainty  quantification,  2014 
SIAM  conference  on  uncertainty  quantification  SIAM  UQ14,  Savannah  (GE),  Mar.  31-Apr.  3,  2014. 
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1.5  Cost  Breakdown 

Hereafter,  I  provide  a  cost  breakdown  of  the  grant  during  the  period  of  performance  (8/1/2014  -  7/31/2015) 

•  PI  effort  -  3.19  months  -  $17,311.94 

•  PI  benefits  -  $5,228.20 

•  International  collaborations  (section  1.3)  -  $9,500 

•  Domestic  Conferences,  Workshops  and  Invited  Talks  (section  1.4)  -  $3,382.19 

1 .  1 1/4/14  -  Invited  talk  at  the  University  of  Massachusetts  Amherst  -  $89.63 

2.  12/4/14  -  APS-DFD  and  invited  talk  at  UCSD  -  $367.81 

3.  2/5/15  -  Invited  talk  at  UCSC  -  $329.87 

4.  6/7/15  -  Invited  talk  at  Stanford  University  and  UCSC  -  $2,445.65 

5.  6/26/15  -  Workshop  at  Woods  Hole  Oceanographic  Institute  -  $149.23 

•  Indirect  Costs  -  $22,138.96 

•  Total  Costs:  $57,561.29 
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